Skip to content

Component benchmarks #93

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 11, 2025
Merged

Component benchmarks #93

merged 1 commit into from
Jul 11, 2025

Conversation

jsturtevant
Copy link
Contributor

@jsturtevant jsturtevant commented Jul 1, 2025 •
edited
Loading

Fixes hyperlight-dev/hyperlight#664

This is a work in progress. The c wit-bindgen component doesn't see to be working correctly.

Todo:

  • get function calls working
  • add build tooling to the scripts (only linux works right now)
  • add to ci workflow
  • general clean up

@jsturtevant jsturtevant force-pushed the component-benchmarks branch from 1c114aa to a6e3164 Compare July 8, 2025 22:36
@jsturtevant jsturtevant force-pushed the component-benchmarks branch from 501720a to 87be2a1 Compare July 9, 2025 02:55
@jsturtevant jsturtevant marked this pull request as ready for review July 9, 2025 03:40
syntactically
syntactically reviewed Jul 9, 2025
View reviewed changes
syntactically
syntactically previously approved these changes Jul 9, 2025
View reviewed changes
Copy link
Member

@syntactically syntactically left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only took a very quick look at this, but it seems sensible.

@jsturtevant jsturtevant force-pushed the component-benchmarks branch from 213f7f5 to a55aed7 Compare July 9, 2025 18:47
@jsturtevant
Copy link
Contributor Author

This is running the new bench marks:

    Running benches/benchmarks_components.rs (/mnt/vss/_work/hyperlight-wasm/hyperlight-wasm/target/release/deps/benchmarks_components-2555538afa605c90)
Gnuplot not found, using plotters backend
Benchmarking wasm_component_guest_functions/wasm_guest_call
Benchmarking wasm_component_guest_functions/wasm_guest_call: Warming up for 3.0000 s
Benchmarking wasm_component_guest_functions/wasm_guest_call: Collecting 100 samples in estimated 6.1145 s (25250 iterations)
Benchmarking wasm_component_guest_functions/wasm_guest_call: Analyzing
wasm_component_guest_functions/wasm_guest_call
                        time:   [246.55 µs 248.92 µs 251.76 µs]
slope  [246.55 µs 251.76 µs] R^2            [0.7427008 0.7392969]
mean   [253.50 µs 258.74 µs] std. dev.      [12.274 µs 14.451 µs]
median [245.04 µs 266.23 µs] med. abs. dev. [4.4610 µs 23.246 µs]
Benchmarking wasm_component_guest_functions/wasm_guest_call_aot
Benchmarking wasm_component_guest_functions/wasm_guest_call_aot: Warming up for 3.0000 s
Benchmarking wasm_component_guest_functions/wasm_guest_call_aot: Collecting 100 samples in estimated 5.0668 s (20200 iterations)
Benchmarking wasm_component_guest_functions/wasm_guest_call_aot: Analyzing
wasm_component_guest_functions/wasm_guest_call_aot
                        time:   [246.63 µs 247.98 µs 249.40 µs]
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high mild
slope  [246.63 µs 249.40 µs] R^2            [0.9059531 0.9055330]
mean   [248.26 µs 251.43 µs] std. dev.      [6.3739 µs 9.6494 µs]
median [246.95 µs 250.05 µs] med. abs. dev. [5.3929 µs 8.3899 µs]
Benchmarking wasm_component_sandboxes/create_sandbox
Benchmarking wasm_component_sandboxes/create_sandbox: Warming up for 3.0000 s

Benchmarking wasm_component_sandboxes/create_sandbox: Collecting 100 samples in estimated 8.4497 s (200 iterations)
Benchmarking wasm_component_sandboxes/create_sandbox: Analyzing
wasm_component_sandboxes/create_sandbox
                        time:   [41.038 ms 41.439 ms 41.876 ms]
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe
mean   [41.038 ms 41.876 ms] std. dev.      [1.6956 ms 2.5673 ms]
median [40.010 ms 41.959 ms] med. abs. dev. [47.406 µs 2.9047 ms]
Benchmarking wasm_component_sandboxes/create_sandbox_and_drop
Benchmarking wasm_component_sandboxes/create_sandbox_and_drop: Warming up for 3.0000 s
Benchmarking wasm_component_sandboxes/create_sandbox_and_drop: Collecting 100 samples in estimated 8.2779 s (200 iterations)
Benchmarking wasm_component_sandboxes/create_sandbox_and_drop: Analyzing
wasm_component_sandboxes/create_sandbox_and_drop
                        time:   [41.502 ms 41.902 ms 42.322 ms]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
mean   [41.502 ms 42.322 ms] std. dev.      [1.7676 ms 2.4514 ms]
median [40.055 ms 42.004 ms] med. abs. dev. [283.84 µs 2.9588 ms]

@jsturtevant jsturtevant force-pushed the component-benchmarks branch 3 times, most recently from 3cd035b to c6c12b2 Compare July 10, 2025 18:26
@jsturtevant jsturtevant enabled auto-merge (squash) July 10, 2025 18:27
@jsturtevant jsturtevant force-pushed the component-benchmarks branch from c6c12b2 to 001cb77 Compare July 10, 2025 18:31
@jsturtevant jsturtevant force-pushed the component-benchmarks branch from 001cb77 to 7e8a5b0 Compare July 10, 2025 22:32
@jsturtevant
Add c Component benchmark
3e3ae13 
Signed-off-by: James Sturtevant <jsturtevant@gmail.com>
@jsturtevant jsturtevant force-pushed the component-benchmarks branch from 7e8a5b0 to 3e3ae13 Compare July 11, 2025 18:32
@jsturtevant jsturtevant merged commit 78f01a0 into hyperlight-dev:main Jul 11, 2025
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ludfjig ludfjig ludfjig approved these changes

@syntactically syntactically Awaiting requested review from syntactically

Assignees
No one assigned
Labels
kind/dependencies
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add a benchmark that matches the existing benchmarks for the modules in HL-wasm
3 participants
@jsturtevant @ludfjig @syntactically